Skip to main content

Week 4

  1. PPT deck
    • Draft ready
  2. NK analysis
    • Do we need any code modifications to obtain this?
  3. LLM with new data by grouping multiple topics together (metadata + value based on time)
    • Will creating JSON strings aggregating metadata and multiple topics help?
    • Does the LLM run as part of the process?
  4. Videos
    • Run yolo on the newly shared videos
  5. txt-to-speech
    • Will run it myself or let Sudhir run it on his laptop (/home/gqc/git/gqc/AI3/audio.py in MSI)

ToDo​

  1. See the Action Items for CCTV project here.

  2. See the Action Items for DeepVibe project here.

  3. Documented exporting python notebook to a script at GQC Python docs.

  • CCTV GIS+Prediction: Porting the latest code to our git repo and testing - ongoing

    • First nb complete 2 and 3 are in progress
    • Add writing to shape files component following the previous lightGBM notebook.
    • Need to save a trained model as currently we only save the optimal hyper-parameters.

  • CCTV-Apps documentation update

    • Docs should be refreshed to reflect the latest status of the apps
    • Draw a mermaid diagram for the dataflow for each app.

  • Document the run details on the binary model training on 'I&I'

    • Now need to copy the training images from the corresponding locations to the MSI (either from Google Drive or downloaded VS_Research folder)
    • Proposed set of updates are present here

  • Setup a rclone copy --update job in MSI once the ext drive with VS_Research is mounted to it

October 26, 2023​

  • Pipe breaks : Clean up the charts with correct labels.
  • Create a presentation for AI3 competetions
    • Include 'first responders' list in the UI
      • GPS coordinates from the people will anyways come as they are wearing sensors.
  • Tried BARD as when official BARD api is out we can use it.
  • Context analysis

October 25, 2023​

  1. Discussed about having a single large DB for all CCTV projects grand cctv db

October 24, 2023​

  1. List the methods applicable for syncing VS_Research folder from Google Drive to MSI

    1. Created a test folder in the Google drive. Inserted some csv files, a subfolder and a zipped file.

    2. Used rclone copy to copy the content to local (MSI)

      rclone copy hydrotrek-gdrive:delete_me_test_rclone_folder delete_me_test_rclone_folder/

    3. Use rclone sync to see if it picks up that there are no changes between remote and local

      rclone sync --dry-run --interactive delete_me_test_rclone_folder hydrotrek-gdrive:delete_me_test_rclone_folder/

      rclone sync picks up the changes between GDrive and local and updates and deletes content in the destination.

    4. Using rclone copy --update

      rclone copy --update --progress  hydrotrek-gdrive:delete_me_test_rclone_folder/ delete_me_test_rclone_folder/

      rclone copy --update only copies the updated files on the source to destination and does not delete existing files on the destination. I think we should use rclone copy --update to perform the sync as if Vannary deletes the VS_Research folder content sync will remove our local copy as well.

      Check the following events:

      # In Google Drive I renamed the `New of cov_output_dataframe.csv` to `New2 of cov_output_dataframe.csv`
      (base) gqc@msi-desktop-ubuntu:WD_BLACK2$ ll delete_me_test_rclone_folder/subfolder_to_test/
      total 60416
      drwxr-xr-x 2 gqc gqc  1048576 Oct 24 14:11  ./
      drwxr-xr-x 3 gqc gqc  1048576 Oct 24 12:43  ../
      -rwxr-xr-x 1 gqc gqc 24964076 Oct 24 12:40 'Copy of cov_output_dataframe.csv'*
      -rwxr-xr-x 1 gqc gqc  2412408 Oct 24 12:40 'Copy of dnv_output_dataframe.csv'*
      -rwxr-xr-x 1 gqc gqc   983976 Oct 24 12:40 'Copy of eve_output_dataframe.csv'*
      -rwxr-xr-x 1 gqc gqc  2412408 Oct 24 12:40 'Copy of normalized_survival_analysis_data.csv'*
      -rwxr-xr-x 1 gqc gqc  2044827 Oct 24 12:40 'Copy of ts_from_devpost_hackathon.csv'*
      -rwxr-xr-x 1 gqc gqc 24964076 Oct 24 14:00 'New of cov_output_dataframe.csv'*

      (base) gqc@msi-desktop-ubuntu:WD_BLACK2$ rclone copy --update --progress  hydrotrek-gdrive:delete_me_test_rclone_folder/ delete_me_test_rclone_folder/
      Transferred:       23.808 MiB / 23.808 MiB, 100%, 11.970 MiB/s, ETA 0s
      Checks:                 8 / 8, 100%
      Transferred:            1 / 1, 100%
      Elapsed time:         1.9s

      # With `rsync copy --update` the `New2` file is downloaded but `New` file is not deleted.
      (base) gqc@msi-desktop-ubuntu:WD_BLACK2$ ll delete_me_test_rclone_folder/subfolder_to_test/
      total 84992
      drwxr-xr-x 2 gqc gqc  1048576 Oct 24 14:17  ./
      drwxr-xr-x 3 gqc gqc  1048576 Oct 24 12:43  ../
      -rwxr-xr-x 1 gqc gqc 24964076 Oct 24 12:40 'Copy of cov_output_dataframe.csv'*
      -rwxr-xr-x 1 gqc gqc  2412408 Oct 24 12:40 'Copy of dnv_output_dataframe.csv'*
      -rwxr-xr-x 1 gqc gqc   983976 Oct 24 12:40 'Copy of eve_output_dataframe.csv'*
      -rwxr-xr-x 1 gqc gqc  2412408 Oct 24 12:40 'Copy of normalized_survival_analysis_data.csv'*
      -rwxr-xr-x 1 gqc gqc  2044827 Oct 24 12:40 'Copy of ts_from_devpost_hackathon.csv'*
      -rwxr-xr-x 1 gqc gqc 24964076 Oct 24 14:17 'New2 of cov_output_dataframe.csv'*
      -rwxr-xr-x 1 gqc gqc 24964076 Oct 24 14:00 'New of cov_output_dataframe.csv'*
      (base) gqc@msi-desktop-ubuntu:WD_BLACK2$

  2. Create a design for seamless integration between CCTV preprocessing and training with external drives involved.

    1. Shared the challenges here

  3. Get familiar with Jira

October 23, 2023​

  • Collecting AgPoint code and documents and draft an email to send them.
    • Get that reviewed by Sudhir
  • Finding the training images for I&I model from dnv 791
Last updated on by Pavan